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OPERATOR-LIPSCHITZ ESTIMATES FOR THE SINGULAR VALUE 

FUNCTIONAL CALCULUS 

FREDRIK ANDERSSON, MARCUS CARLSSON, AND KARL-MIKAEL PERFEKT 


Abstract. We consider a functional calculus for compact operators, acting on the singular values 
rather than the spectrum, which appears frequently in applied mathematics. Necessary and 
sufficient conditions for this singular value functional calculus to be Lipschitz-continuous with 
respect to the Hilbert-Schmidt norm are given. We also provide sharp constants. 


1. Introduction 


For simplicity we restrict attention to the finite-dimensional matrix case in this introduction. Let 
A be a matrix with singular value decomposition A = USV* , and consider the operation of changing 
the singular values by applying some function / : R_|_ —>■ C to E, thus yielding a new matrix which 
we will call /s(A), where the subscript s indicates that we are considering a singular value functional 
calculus. For matrices with non-trivial nullspaces, it is easy to see that the condition /(O) = 0 is 
necessary for fs{A) to be well defined (Section [2]). Let us also remark that, in case / is a function 
defined on C and A is a normal matrix, then /(A) is defined by the classical functional calculus 
(CFG) based on the spectral theorem. However, it is rarely the case that /(A) = /s(A) except when 
A is positive (Section [5]). 

The operation A i—>■ /s(A) is commonly seen in applied mathematics, since it often appears as the 
proximal operator |18| in Matrix Optimization Problems and Compressed Sensing. For applications 
in Computer Vision, Structure from Motion, Photometric Stereo and Optical Flow, see [mini and 
the references therein. See |7] for its use in alternating projection schemes and [5] for a problem 
in financial mathematics HU. For applications in Control Systems see m. MRI see [H], and for 
applications to complex frequency estimation see [3]. More examples can be found in Elllz]- 

When studying convergence of algorithms utilizing the singular value functional calculus, it is 
important to have bounds for the distance ||/s(A) — fs{B)\\F given ||A — RUf, where the subscript 
F indicates that we are dealing with the Frobenius norm, (which is the same as the Hilbert-Schmidt 
norm || • but we will follow standard conventions and use this notation only for the infinite¬ 
dimensional case). We thus define 


( 1 ) 


||/s||Lip = sup 

A,B 


\\fs{A)-UB)\\F 

\\A-B\\f 


In Section |4] we shall show that this supremum turns out not to depend on the dimension of the 
matrices A, B, which is why this is omitted from the notation. 

If one restricts attention to positive matrices in the above supremum, then it is known that it 
equals II/IILip- This follows from more general work concerning the CFC-case [12], a result which 
is also presented with a very simple proof in jS] Lemma VH.5.4] and was rediscovered in |5D]. We 
remark that in the CFC-case, results concerning Holder and Lipschitz continuity with respect to 
various operator norm have a long history, see e.g. mum and the references therein. 

The main result of this paper is the following: 
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Theorem 1.1. Let f : R_|_ —>■ C 6 e continuous with /(O) = 0. Then ||/s||Lip < v^H/llLip the 
constant \/2 is the best possible (if it is to hold for all Lipschitz functions f). However, if f is real 
valued, then 

ll/.lkip = ll/llLip. 

For an interesting related result also having \/2 as the best constant in the complex case and 1 
in the real case, see [4]. In terms of applications, the study of real valued functions is most relevant. 
Based on general arguments [ 8 ], one can show that ||/s||Lip should be bounded in terms of ||/||Lip, 
but the fact that the constant is 1 is rather surprising and is likely to have an impact on algorithmic 
design involving the singular value functional calculus. 

2. The singular value functional calculus 

Let H be a separable Hilbert space of dimension d, 1 < d < oo, and suppose that A : TL ^ TL is 
a compact operator, A G Bq. Then A has singular value decomposition; there exist an orthonormal 
basis (un)n=o of H and an orthonormal sequence {vn)n=o ®^oh that 

d 

(2) A= y^^Sn(A) U„l8Vn, 

n—0 

where s„(H) are the singular values of A. In other words 

d 

Ah='^ Sn{A){h,Vn)Un. 

n—0 

An equivalent formulation of ([1]) is that A has a polar decomposition A = C/E, where C/ is a partial 
isometry and E = |A| is a positive diagonalizable operator such that Eu„ = Sn{A)vn- We will 
primarily be concerned with operators A in the Hilbert-Schmidt class 52 , i.e. compact operators 
such that 

d 

II^IIS2 = 

n—0 

Following standard conventions we denote this norm by ||A|||. whenever d < oo, in which case it 
coincides with the ^^-norm of the elements of the matrix representation of A in any orthonormal 
basis. 

Given any continuous function / : K_|_ —>■ C such that /(O) = 0 we define /s : Hq —t Bq by 

OO 

(3) fs{A) = f{Sn{A)) Un®Vn, 

n—1 

or equivalently that fs{A) = Uf{Y,), where /(E) is the operator such that /(E)u„ = /(sn(A))u„. 
The subscript s indicates that we are dealing with a “singular value functional calculus”. To see that 
it is well defined, note that if s = s„(A) 7 / 0 and h G ker(s^J — A*A), then 

UA)h = ^Ah. 

However, note that if s = 0 and we were to allow /(O) 7 / 0 it is clear that fs{A)h could depend on 
the particular choice of (u„)n and (u„)„. 

We remark that / only needs to be defined on R+ for fs{A) to exist, and obviously 

fs{A) = f{A) 

for all positive operators A. For normal operators, the situation is more complex. Consider for 
example 
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which has singular value decomposition UT,V* = All. Then 

MA) = AUDI = ( ) 

whereas f{A) is not even defined in an the classical functional calculus, due to the negative eigenvalue 
— 1. Moreover, if / is defined on R we clearly have f{A) ^ fs{A) unless /(I) = —/(—I). This is 
further highlighted by the next proposition. 

Proposition 2.1. Let / : C — >■ C be a continuous function with /(O) = 0. Then fs{A) = f{A) for 
a normal compact operator A : TL ^ TL if and only if f satisfies 

for every non-zero eigenvalue A of A. 


Proof. Since A is a compact normal operator, there is an orthonormal basis (vn)n of Tl such that 

OO 

(4) ^ A„ i;„ (g) Vn, 

n—1 

where A„ are the eigenvalues of A, implicitly omitting the zero eigenvalues from the sum (U). On 
the other hand, an SVD of A is given by 

OO 

A = ^ ^ I Xn I Il-n O Vn , 

n—1 


where Un 


An/|A„|i;n- Therefore, we obtain that 


f{A) = /(A„) Vn O Vn 

n—1 


while 

r ^ Xnf{\Xn\) 

JAA) = 2^ —ryn— Vn 0 V, 
n—1 ' 

proving the proposition. 


□ 


Corollary 2.2. Let p be a polynomial without constant term. Then Ps{A) = p{A) for all normal 
compact operators if and only if p{z) = az, a G C. 


3. Complex doubly substochastic matrices 

This section contains the main technical tool for the proof of Theorem ll.il We say that a square 
matrix is complex doubly substochastic (cdss) if for each row and column, the £^-sum of entries is 
less than or equal to 1. (In [19], such matrices are simply called doubly substochastic. However, 
most other sources using this term include a non-negativity condition on the elements, which is 
why we have chosen to clarify by adding complex.) Our main interest in cdss-matrices stems from 
the fact that if U and V are unitary matrices then U QV is cdss, where © denotes the Hadamard 
product, as follows immediately by the Cauchy-Schwarz inequality. 

Let TT denote any permutation of length n and let 7 be a vector of the same length containing 
unimodular entries. We denote by the n x n matrix whose (j, 7rj)’th value is 7^, all other 

entries zero. The following lemma is likely known, but lacking a reference we provide a simple proof 
based on the Birkhoff-von Neumann theorem, (see e.g. |5]). 

Lemma 3.1. An n x n matrix is complex doubly substochastic if and only if it lies in the convex 
hull of : 71,7}. 
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Proof. Let V denote the set of cdss n x n-matrices. It is clearly a closed convex set. We shall show 
that the extreme points of V are precisely the matrices of the form for some permutation 

TT and vector 7 . The lemma then follows immediately by the Krein-Milman theorem (or rather, 
Minkowski’s theorem on convex sets na, since we are in Euclidean space.) 

First, let tt and 7 be given. Let denote the ij:th entry of Suppose that = 

{A + B)/2 for matrices £ V with entries and bij, respectively. Suppose that = 1. 

Since \aik\ < 1 and \hik\ < 1 this forces that atk = bik = rriik and hence that = btj = 0 for 
^ ^ j ^ n, j k. Therefore A = B = and thus is an extreme point of V. 

For the converse, let M £ V, with entries rriij, be an extreme point. Consider first the case where 
M has a row with sum of absolute values strictly less than 1, say the p :th row. Then clearly 

n n 

< n, 

i=i j=i 

which, upon changing the order of summation, shows that there is also a column with a sum of 
absolute values strictly less than 1, say the q:th column. Let E be the matrix with its pq-.th entry 
e, all other entries 0, and let A = M — E and B = M + E. For sufficiently small e we find that A 
and B are cdss, which contradicts the fact that M is an extreme point of V, since M = {A + B)/2. 

We have thus shown that the extreme point M has sums of absolute values of all rows and 
colums equal to f. In other words, the matrix Tfl with entries \mij\ is a doubly stochastic matrix. 
By the Birkhoff-von Neumann theorem, Afl is either of the form for a permutation tt and 

7 = (f, f,..., f) or it is not an extreme point and thus of the form = (A + B)/2 for two doubly 
stochastic matrices A and B, A B. In the former case, M is of the form M,r ,7 for a suitable 
sequence 7 . In the latter case, as seen by adjusting the arguments of the entries of A and B, M is 
clearly not an extreme point of V, a contradiction. □ 


4. Operator Lipschitz estimates 


Throughout we let / : R+ —>■ C be Lipschitz with /(O) = 0. Let Ai,A 2 £ C be two scalars 
interpreted as operators on 'H = C. Then /s(Aj) = p^/dA^j) for j = 1,2. Hence a Lipschitz 
condition \\fsiA) — fsiB)\\s 2 < C\\A — BWs^ implies that 

( 5 ) |ci/(a:) - C2/(2/)| < C|cix - C2y| 


for all x,y >0 and Ci,C 2 £ T, where T denotes the unit circle in C. This motivates the following 
definition 


( 6 ) 


||/||Lip-C= sup 

cGT 


\f{x)-cf{y)\ 
\x - cy\ 


Proposition 4.1. Suppose that f : K+ —>■ C satisfies /(O) = 0. Then 

(7) ll/||Lip-C<^||/||Lip 
where \f2 is optimal. If f is real-valued, f : R+ —>■ R, it holds that 

(8) ll/lkip-c = ll/lkip 

Proof. We may clearly assume that ||/||Lip = 1- Suppose first that / is real-valued and write 
c = a -\- ib. Noting that the hypotheses imply that |/(a^)| < x for all x, we have 

\f{x) - c/(y)p = |/(x) - /(y)p -f 2(1 - a)f{x)fiy) < |x - y\^ -f 2(1 - a)xy = \x - cyp, 

which shows that ||/||Lip-c < ||/||Lip- Since the reverse inequality is obvious, ([5]) follows. 

For / complex-valued, the computation is more involved. The inequality ([7]) is clearly equivalent 
with 


( 9 ) 


l/(a;) - c/(y)| < \/2|x - cy|, x,y£R+, c £ T, 
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(still assuming that ||/||Lip = !)■ Fix c and suppose without loss of generality that y < x. If 
f{v) = 0 there is nothing to prove since \x — y\ < \x — cy\. Similarly, if f{x) = f{y), ([HI) follows from 
the fact that |/(y)| < y. If f[y) ^ 0 and f{x) ^ fiy), let in G C be such that f{x) = wf{y). Since 
|/(a;) — f{y)\ = lin — l||/(y)| <x — y there is a constant d, 0 < d < 1 such that 


Since \f{y)\ < y it follows that 

( 10 ) 


l/(y)l = d 


x-y 
Iw- I|' 


X < 




y- 


It is straightforward to check that the function x i-a {x — y)/\x — cy\ is increasing when a; > y, which 
combined with (flUl) yields that 


x-y 


X 


- cy| 


< 


■-M/d 


l + tjH-e 


< 


\w - I| 


d|l + Iw — I| — c| 


Recalling that f{x) = wf{y) we hence conclude that 


\fix) - c/(y)| = Iw - c\d -^—^ < |w - cl 


' - cy| 


k - 1 | 


1 + k - I| - c| 


We claim that 

( 11 ) 


sup 


\w — c\ 


6 C.|c| = l|l+k-l|-c| 


= V2. 


From (fTT|) we immediately deduce ([5]), and it also implies that y/2 is the best possible constant. To 
see the latter, hx ru G C and a unimodular c and set x = 1 + k ~ 1| y = 1- We may then dehne 
a function / as in the statement of the theorem satisfying that /(I) = I, /(! + k“ 1|) = which 
gives 

\f{x) - c/(y)| ^ \w-c\ 

|a;-cy| ll + k-ll-cj' 

To see that the supremum of m is at least v^, let w = 1 — it and let c = e*‘. For this choice 
of w and c we have that 

k — c| |1 — e** — it\ /- 

|l + k-l|-c| |l-e**+t| ’ 

For the upper bound, note that upon squaring it is equivalent with 

k — cp < 2 |i + k ~ i| ~ cp, 

which upon expanding, reordering and noting that |c| = 1 is equivalent with 

kl^ + 5 — 4Re w + 4k ~ 1| “ 2Re [c(2(I + k “ 1|) “ 'f^)] ^ 0- 
This inequality is true for all unimodular c if and only if 

(12) kP + 5 — 4Re w + 4k “ 1| ~ 2|2(1 + k “ 1|) ~ ^^1 > Oj € C. 

Since |2(I + k ~ 1|) — w| < 2k — 1| + k “ 2| by the triangle inequality, (fT^ is implied by 

kP + 5 — 4Re w — 2k — 2| > 0, w e C. 


The left hand side equals 


k - 2p + I - 2k - 2| = (k - 2| - l)^ 


which completes the proof. 


□ 
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The next theorem is the key result of the paper. Theorem II.11 is an immediate corollary of this 
result and ProDOsition l 4 .ll Set 


(13) 


||/s||Lip= sup 

A.,B^S 2 


\\A-B\\s, 


It will also follow from the proof that the definition ([T]) is independent of the dimension, as claimed 
in the introduction, and that it coincides with (I13II . 


Theorem 4.2. Let / : R+ —>■ C &e Lipschitz with /(O) = 0. Suppose that A,Bg 82 - Then fs 
satisfies 

( 14 ) \\fs{A)-MB)\\s, < WfWup-cU-BWs, 

and this estimate is optimal. In other words, ||/s||Lip = ||/||Lip-C- 

Proof. We first prove m for hnite square matrices. That is, we will show that for d x d matrices 
A and B we have 


(15) ||/.(^) - /.(B)IIf < ll/llLip-cll^ - By. 

Suppose that this has been proved and consider general A, B G S 2 with singular value decompositions 

00 00 

A = J2sn{A)u^i^vy B = ^Sn{A)u^®V^. 

n—1 n—1 

For N > 1 we may consider 

N N 

An = ^Sn{A)u^®vy B =^Sn{,A)u^®V^. 

n—1 n—1 

to be operators acting on Vn = span{rt^,| 1 < R < LI}- Note that the singular value 
decompositions of Aat and Bn are identical whether considered operators on B or on Vn, and 
that the exact same statement applies to fs{AN) and fs{BN). Hence (fTKll applied to the finite¬ 
dimensional space Vat gives us that 

||/s(^Ar) - fs{BN)\s 2 (H) < ll/llLip-cll^Af “ HatHs^ch). 

Since HHtv — A\\s.^[-h) 0 and ||/s(HAr) — /s(H)|| 52 Ch) —t 0 as —>• 00 , and similarly for B, the 

inequality (fH)) follows. 

We now turn to proving (|15l) for d x d-matrices A and B, for which we express the singular value 
decompositions with the usual matrix notation; 

A = Ua^aVX, B = Ub^bVX. 

We have the following formula for ||H — il||p, 

||H - By = WUa^aVX - Ub^bVXWI = \\U*bUa^a - 

= ||H||| + \\By - 2Re^ © U*bUa^a, 

ae 

where JXae denotes the operation of summing all entries of a matrix, and M denotes the action of 
taking the complex conjugate of every entry of a matrix M. 

Since V^Va 0 UgllA is cdss. Lemma [3.11 implies that there exist ci,... ,cm in [0,1] satisfying 
^^1 c„ = 1, permutations 7r„ and length-d vectors 7 „ with unimodular entries, 1 < n < M, such 
that 

N 

vyy 0 u*bUa = Y. 

n—1 
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We get 


M 

||A - BWl = Pill + ||B||| - 2Re ^ c„ ^ 

n—1 ae 

M / N N N \ 

= '^Cn ( - 2Re^Si(-B)7„,iS^„ .(A) j 

n—1 \z=l i—1 i—1 / 

M N 

= ^ Cn'^\Si{B) - 7n,iS77„,i(^)|^- 

n—1 i—1 

This identity applied to fs{A) = and fs{B) = UBf{^B)yB immediately gives 

M N 

\\fs{A) - UB)\\l = E E - 7nP(s.„,P))P 

n—1 i—1 


Since \ f{x) - jf{y)\ < ||/||Lip-ck “ 72/1 for every x,y>0 and 7 e T, we get 


M N 

WUA) - UB)\\l = E E - 7nP(s.„7A))p < 

n—1 i—1 

M N 

^ ll/llLp-C E E = ll/llup-cll^ “ 

n—1 i—1 

which establishes (HI. The optimality of (1141) follows immediately from the argument surrounding 
This also shows that ||/s||Lip = H/Huip-C independent of whether we use definition ((T51) or ([T]) 
with arbitrary fixed dimension. 

□ 
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